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Dear Sir: 

In support of the accompanying response to the February 20, 2008 office action in the 
above-referenced application, I hereby declare as follows: 

1 . I am the first-named joint inventor of the subject US patent application 
(hereinafter "the Application"), which is assigned serial number 09/614,363 and was filed on 
July 12, 2000. 

2. I am submitting this declaration to show that some details of my invention 
claimed in the Application are described in US patent number 6,567,083 (hereinafter "the 
Patent"). In other words, using the language of MPEP 715.01(c), the Patent is a publication of 
my own invention. To that end, this declaration includes facts showing that: 

• I jointly made the invention upon which the relevant disclosure of the Patent 
is based, 

• I was associated with Daniel Baum, the first named inventor of the Patent, and 

1 



Attorney Docket No.: 2839/1 15 

• Mr. Baum derived his knowledge of the relevant subject matter from me. 

I jointly made the invention upon which the relevant disclosure of the Patent is based 

3. In 1996, 1 was employed at Silicon Graphics, Inc. ("SGI") in Mountain View, CA 
as a staff engineer. In the summer of 1996, 1 started work on building a machine that used 
floating point rasterization, including floating point scan conversion, and a floating point 
framebuffer. The internal code name for the machine at SGI was ''Bali." The Application 
describes and claims various aspects of Bali. 

4. The general architecture of Bali is described in the docimient entitled "Bali 
Offsite," which is dated November 12, 1996 and attached as Exhibit A. This document was 
discussed among SGI persoimel at an off-site meeting on November 12, 1996. Among other 
things, this document shows: 

• A reference to the "R Chip" on the Agenda page (immediately following the 
cover page), the Contents page, and page 22. The R Chip refers to the portion 
of the Bali device performing rasterization, 

• A chart of software simulating Bali on page 2 1 , 

• A block diagram of the rasterizer (i.e., the R Chip) and scan converter on page 
24, 

• a block diagram of a floating-point block on page 29, and 

i 

• floating-point multipliers and floating-point normaUzers/adders on page 35. 

5. A number of other documents show specific details and aspects of my invention. 
These documents were prepared either by one of the joint inventors of the Application or by 
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someone with information that originated from one of the joint inventors. Some of those 
documents are discussed paragraphs 6 to 13. 

6. "Bah Floating Point Representation" - A document dated December 3, 1 996 
discussing a 16-bit floating point format and internal variations used inside the R chip pipeline, 
and submitted herewith as Exhibit B. 

7. "Mapping RenderMan to OpenGL" - A document I authored, dated January 24, 
1997, that includes software code to be used with Bali hardware to have floating point processing 
through the R Chip pipeline, and submitted herewith as Exhibit C 

8. Document containing Code headed "head 1.1" - A document I authored, dated 
January 24, 1997, that describes rasterization of pixels, color processing using floating point 
numbers and framebuffers, and submitted herewith as Exhibit D. 

9. "Extended Range and/or Precision" - A document dated February 15, 1997 and 
authored by my co-inventor, Mark Peercy, and me. This document describes calculations using 
floating point numbers and which format of floating point numbering to use, and submitted 
herewith as Exhibit E. 

10. "Tiny Floating Point" - A document dated February 1 5, 1 997 and authored by 
Mr. Peercy and me. This document describes disadvantages of fixed point arithmetic and further 
describes a tiny IEEE like floating point format, and submitted herewith as Exhibit F. 

1 1 . Document containing Code headed "head 1.5" - A document with versions 
ranging from dates from July 15, 1997 to July 28, 1997 and authored by John Paul Alex, who 
was an intern working for Mr. Peercy and me. This document describes calculations using 



3 



Attorney Docket No.: 2839/1 15 



floating point numbers and which format of floating point number to use, and submitted herewith 
as Exhibit G. 

12. Email from me to Mr. Baum dated August 14, 1997. In this e-mail, I mention the 
slOeS pipeline and high-speed data transfer from the host to the frame buffer, the frame buffer to 
the texture memory, and texture memory to the frame buffer. This document is attached as 
Exhibit H. 

1 3 . "Invention Disclosure" - A document dated September 22, 1 997 that is an 
invention disclosure submitted to SGI Legal Services, and submitted herewith as Exhibit I. The 
document describes a 16 bit floating point format for texture store and framebuffer. The 
document further states that the invention was conceived about a year prior to the submission of 
the Invention Disclosure. This document also notes that Mr. Baum is the department manager 
for visual systems. Mr. Baum is not listed as an inventor. 

14. Mark Leather, who was a member of the SGI technical staff from 1989-1997 but 
not an inventor on the Application, further corroborates my invention of the subject matter. 

1 5. During a deposition conducted on December 7, 2007 (see Exhibit J), when asked 
if it was his "understanding that [the idea of multipassing data through the frame buffer] 
involved the use of floating point formatted data," Mr. Leather responded: "Yes, that was my 
understanding." 

16. Further, in response to the question "And did you understand that that was [Mark 
Peercy's] concept that he was working on at SGI?," Mr. Leather responded: "I believe it was him 
and John Airey." 
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1 7. Diiring the deposition, in response to the question "But do you recall the use of a 
floating point frame buffer as it related to Bali?" Mr. Leather responded: "Yes." Further, in 
response to the question "And that was Dr. Peercy and Dr. Airey's work?" Mr. Leather 
responded: "Yeah." 

18. During the deposition, Mr. Leather recollected that a second project at SGI had a 
floating point frame buffer, but when asked if the "other work was an extension of Airey and 
Peercy' s work," Mr. Leather replied: "Yeah." 

I was associated with Daniel Baum at the time of my joint invention 

1 9. Mr. Baum was my supervisor as the Hardware Director for the Bali project. We 
thus both worked together at SGI at the same time — among other times, while I was developing 
the invention claimed in the Application. 

Daniel Baum derived his knowledge of the relevant subject matter from me 

20. hi his capacity as hardware director of the Bali project, Mr. Baum was responsible 
for understanding the technology and, as such, he was a decision-maker on the direction, 
schedule, and features of the project. In addition, in this capacity, Mr. Baum was responsible for 
updating company executives. To fulfill these and other responsibilities, Mr. Baum needed to be 
familiar with the software simulation, the floating-point scan converter, the floating-point frame 
buffer, and their technical advantages/benefits. 

2 1 . My co-inventors and I therefore described various aspects of the invention and the 
simulation to Mr. Baum so he could make the appropriate decisions. For example, Mr. Baum 
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was present at the above noted Bali offsite meeting of November 12, 1996 (see Exhibit A), as 
well as other internal Bali meetings. In addition, in his capacity as Hardware Director of Bali, 
Mr. Baum had access to many of the documents discussed herein. 

22. As noted above, on August 14, 1997, Mr. Baum sent me an e-mail asking about 
opportunities to differentiate SGI. I responded, as noted above, by discussing the slOeS pipeline 
and floating-point frame buffer (see Exhibit H). I thus conveyed details of my invention to Mr. 
Baum in this e-mail. 

23. Daimy Loh, one of the joint inventors of the Application, worked on the software 
simulation that appears to be mentioned in the Patent. Specifically, during a deposition 
conducted on November 9, 2007, Mr. Loh stated "So the question you asked me before, did I 
work with John Airey on the floating point on Bali?.... Yes." Mr. Loh further stated that he 
"wrote sort of the simulation framework to evaluate floating point formats in the frame buffer." 
When asked if Mr. Loh was doing that work with John Airey, Mr. Loh replied "with John Airey 
and Mark Peercy." That portion of Mr. Loh's transcript is in the prior noted Exhibit J. 

24. In his capacity as Hardware Director, Mr. Baum should have known about the 
software simulation. 

25. It is my understanding that before the filing date of the Patent, SGI did not have 
another floating-point project that was independently derived. Instead, as Mr. Leather stated 
under oath, if any such projects existed, they were derived from the principles of my joint 
invention. 
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26. I hereby declare that all statements made herein of my own knowledge are true, 
that all statements made herein on information and belief are believed to be true, and further that 
these statements were made with the knowledge that willful false statements and the like are 
punishable by fine or imprisonment, or both under Section 1001 of Title 18 of the United States 
Code and that such willful false statements may jeopardize the validity of the application or any 
patent issuing thereon. 




Dated: 
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Vector Processor 



[VReg^ 
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Vector 
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Scalar 
Unit 



VP = IM gates, 

VU = 250K gates, 

DMem = 5.8K gates, 

VI$ = 6.3K gates. 



170K bits, 
9K bits, 
384K bits, 
134K bits. 



272 sq mm 
34 sq mm 
24.4 sq mm 
15.34 sq mm 
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PE Design Notes 

Rob (Skip) El-Kareh 
Gloria (Globot) Lau 
Erik (Bowzer) Lindholm 
Paul (Ironman) Thilking 
Mark (Flyboy) Young 



GE 



PE 



input: lOK gates, 2SK bits 
pass: 6K gales, OK bits 
pixel: 23K gates, OK bits 
geom: 283K gates, 28K bits 
output: 92K gates, OK bits 
misc: 28K gates, OK bits 
Total: 442K gates, 53K bits 



PE/G 



N 



PE / Top Level Block Diagram 



November 11, 1996 4:05 pm 
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Vertex 
Buffers 
(64 Words) 
(6 Bufiers) 



Primitive 
State Machine 
(Vertex Selector) 






BackfaceCull 
(PolygonOffset) 







LineStipple 
(Major Length) 
(True Length) 



Primitive J CUp 
State Machine Vertex 
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Geometry Block / Internal Block Diagram 
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PE Functional Goals 

• GE interface to Omega Network 

- Includes interface to Convert/Merge 

• Pixel conversion, reformatting, and 
distribution 

- Input (IEEE Float, U16, U8), start 
aligned, no garbage 

• Output (S10E5, U16, U8), first pixel word 
aligned start of each packet . 

• Bitmap unpacking, reformatting, and 
distribution, non-opaque and opaque, pixel 
packet per bit 



• Geometry primitive processing and 
distribution 

- Standard OpenGL primitives (withswap) 

- PoIygonMode, line and point generation 

- PolygonOffset, dz/dx & dz/dy, with scale 
and bias (may incur slight penalty) 

- LineStippIe counter, major and true 
lengths, with first vertex of each segment 

- Clip exception processing (sent to GE, 
-p WILL incur sig^ficant penalty) 

- Backface cull before clip ^t^^^^^ 

- Zmin/Zmax/Hitflag for select 

- Bounding box coverage generation per 
primitive (including lines) ffl^CU^ 



■ Feedback on Host or 



inGEl 



• Rasterizer State Management 

- Write /WriteThrough 

- Dirty Bits per State Set 

• R Chip Synchronization 

- Broadcasts SyncID to R chips when 
Geometry is received before SyncFIag is 
cleared 



> Assumptions 

- R Handles: stippled line rasterization, 
wide stippled lines, wide points, AA lines 
and points 

- Bounding Box rasterizer selection is good 
enough for lines 




Open Issues 



Who handles rasterization of Constant 
Multisample Points (Lightpoints) 

What is the true benefit of R state 
shadowing? What is the state size? Can it 
be grouped into sets? Could it be better 
managed in N Chip? 
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Packet Ordering 

Only one path from Node A to Node B within 
Omega network, so packets on any one channel 
arrive in order. 



Most transfers are self-descriptive and do n( 
about ordering: 

Texture requegls 



Others will have to cope using sequence numbers. 



System-Level Deadlock 



Virtual channels (sharing the same wires) is 
traditional way to allow high-precedence messages 
to flow despite snari of low-precedence messages. 

Virtual channels Inriply: 
Separate buffering 
Separate flow control 
Intemiingling between virtual channels 
(each transfer identifies its virtual channel) 

We need to idenUfy all possible deadlocks! 



High Medium Low 

Video request Texture request Triangles 

Video response R ♦ G Throttle Veilexes 

Texture response Pixel read data Read requests 
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Sometimes node cannot accept packets. 
Example: R-chip can be ovenvhelmed by 
texture requests or pixel reads or triangles. 

Need to be able to refuse packets. 



rhrottle packets from R-^G. 

Others handled by flow control wires. •*■ oAC ^er 

Two schemes: 

Throttle: shut up ASAP. 

Credit: keep track of receive buffers 

Credit is more effective...hldes the latency...but 
it presumes one set of receive buffers. If we have 
multiple FIFOs behind each input, credit scheme 
gets complicated. 



Error Handling 

hronous signa 
unless it is tuned perfectly. 

During bringup, "perfect tuning* can take months 
to achieve. Error con-ection can allow this effort 
to proceed in parallel with debug, not in series. 



Before we choose, we need to understand wfiat 



N Chip Estimate 



10 input pons, 10 output pons, 16-bil/port 



Eiror handling (error detection and oorredkm) 

Peifonnance monitor 

dlagnoseolocao 

Output unit (SSD, oubut aititratlon) 
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Bali Simulation 



Define Design 
Understand Risks 

Interface Review Design Review 



Put It Together 
Sanity Check 



Exhaustive Verification 
Push Physical Design 



Feature Set Defined 
■I 

Set Design Rules 

I— i 

Set Coding Style 
^ Write Cliip Specs 
Set Chip interfaces 



% Functiorjaiity 
30% Correct • 



Design Review 



1Q0% Functionality 
90% Correct 



Regressions on Asim 



Set Ctiiplet Interfaces ; 

»~ * tr.^l j 

Write Higti Level Sim (timini g, layout^ i 

V/rite Chiplet Tests 



Reqressions on RTL 



Set Verification Rules 

1— I 

Write Chip Test Specs ^ 

Write System Test Specs 

Make Test Jig Creator Tool 
I ^ — I 



^Chiplet Testing 



Chiplet Bug Fixes 



Write Synthesizable RtL, Timing and Layout 



Asim Bug Fixes 



RTL Bug Fixes 



Test Jig Creation ; 



1 

^ Write Chip Sanity Tests 



■ Write System Sani ty Tests 



"IntensiveTest" writing; 



Write Fastsim C Simulator ■ 

I 1 I 

ucode and system sv/ developnient 



Period (Asim and RTL) 



Chip Verification in As^ and RTL 



Syste m Verification in Asim and RTL 
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LTF Block Diagram 



TFI 




to/ti 





]^rdy 



Scan Converter . . 

(®y®) (screen) 
bf N L H Att ic xyz.a» xyz,slope mask 



I ♦ t ♦ ♦ 



1 



rdy 



Vector Unit (VU) 
tO/t1 bf SpotL N.L (N.H)^n 



Lighting Unit (LU) 
tO/t1 Cs 



Texture Env (TEVO) 



I 



Texture Env (TEV1) 



Fog 



Alpha & l\/laslc 



color, mask, XYZ, dZdX, dZdY, exp 
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Lighter Block Diagram 

L tO/t1 N H tO/tl L tO/tl Att 



tO/t1 - 
ic - 



tO/t1 
ic ■ 



tO/t1 ■ 
ic ■ 



bf 



Vector Unit 



C 



II 



S 



exp 



it- 



N L I I (N.H)^n 1 I (S.-L)^exp | 




Lc (to TEV) 



Lc = Att*SpotL*CI { // distance^spot light attenuation*light color 
Am // ambient material*ambient light 



} 



+ Dm*(N.L) // diffuse materiardiffuse light 
+ Sm*(N.H)^n // specular material*specular light 



ic = interpolated color 
tO/t1= textures 
*_f=front material color 
*_b=back material color 
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Texture Environment Block Diagram 
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C = A*Cf + B*Ct + D 



Ct: texture color 
At: texture alpha 
Cf : lit fragment color 
Cc: constant color 
Cb: bias color 



Replace Fragment: 

Replace Texture: 

Modulate: 

Blend: 

Decal: 

Add 



A= 1 
A= 0 
A= Ct 
A=-Ct 
A=-At 
A= 1 



B= 0 
B= 1 
B= 0 
B = Cc 
B= At 
B = Cc 



D= 0 
D= 0 
D= 0 
D = Cf 
D = Cf 
D=:Cb 
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Fog Block Diagram 
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Alpha and Mask Block Diagram 
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omega net 
interface 



rspjbusy 



256 entries 
32-bits 



Request 
?SM 



re<iuest_val id 



current_request 



RAM: 256 x 32 bits for FIFO 
FFLOPSi 64 current_req:uest 

9 f ifo rd pointer 

9 f if o wr pointer 

1 state bit for FSM 

1 request_valid 

1 rsp_busy 

Clock coTint: 1 to get to front of FIFO, 

1 to get out of FIFO into Request FSM 

2 in the FSM. 
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